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Abstract 

We propose new primal-dual decomposition algorithms for solving systems of inclusions involving 
sums of linearly composed maximally monotone operators. The principal innovation in these al¬ 
gorithms is that they are block-iterative in the sense that, at each iteration, only a subset of the 
monotone operators needs to be processed, as opposed to all operators as in established methods. 
Deterministic strategies are used to select the blocks of operators activated at each iteration. In 
addition, we allow for operator processing “lags”, permitting asynchronous implementation. The 
decomposition phase of each iteration of our methods is to generate points in the graphs of the 
selected monotone operators, in order to construct a half-space containing the Kuhn-Tucker set as¬ 
sociated with the system. The coordination phase of each iteration involves a projection onto this 
half-space. We present two related methods: the first method provides weakly convergent primal 
and dual sequences under general conditions, while the second is a variant in which strong conver¬ 
gence is guaranteed without additional assumptions. Neither algorithm requires prior knowledge 
of bounds on the linear operators involved or the inversion of linear operators. Our algorithmic 
framework unifies and significantly extends the approaches taken in earlier work on primal-dual 
projective splitting methods. 
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1 Introduction 


This paper considers systems of monotone inclusions of the following general form. 

Problem 1.1 Let m and p be strictly positive integers, set I = m} and = {1,... ,p}, and 

let {%i)i^i and {Gk)keK be real Hilbert spaces. For every i G / and k G K, let Ai\ T-Li — )• 2^* and 
Bk- Gk ^ 2^'= be maximally monotone, let z* G Hi, let G Gk> and let L^i: Hi ^ Gk be linear and 
bounded. Consider the coupled inclusions problem 



find {xi)i^i G ^^Hi such that (Vi G/) 


( 1 . 1 ) 


i&I 

its dual problem 


find {vDkdK G Gk such that 


k&K 



and the associated Kuhn-Tucker set 



k&K 



The problem is to find a point in Z. The sets of solutions to (1.1) and (1.2) are denoted by and 3>, 
respectively. 

As discussed in [1], Problem 1.1 models a wide range of problems arising game theory, image 
recovery, evolution equations, machine learning, signal processing, mechanics, the cognitive sciences, 
and domain decomposition methods in partial differential equations. In [15, Section 5], it was shown 
that an important special case of Problem 1.1 is the following optimization problem, in which the 
monotone operators {Ai)i^i and are taken to be subdifferentials. 

Problem 1.2 Let m and p be strictly positive integers, set / = {1, ..., m} and K = {1,... ,p}, and let 
{Hi)i^i and {Gk)k£K be real Hilbert spaces. For every i G I and k G K, let fi'.Hi ^ ]— oo, +oo] and 
gk - Gk ^ ]— oo, +oo] be proper lower semicontinuous convex functions, let z* G Hi, let rk G Gk, and 
let Lki: Hi ^ Gk be linear and bounded. Suppose that 



(1.4) 


The problem is to solve the primal minimization problem 



(1.5) 
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along with its dual problem 

, . ^ '12 (^i ~ Y1 + Y1 isk{4) + {4 I rk)). (1.6) 

In recent years, several decomposition algorithms have been proposed to solve Problem 1.1 (or 
at least the primal problem (1.1)) under various hypotheses [1, 2, 3, 9, 10, 11, 15, 16, 17, 25], In 
such algorithms, the monotone operators as well as the linear operators are evaluated individually. 
The methods we propose in the present paper for solving Problem 1.1 are based on those of [1, 2], 
which are themselves based on the projective primal-dual methods initiated in [19, 20] for finding 
a zero of the sum of monotone operators. The basic idea underlying this class of methods is to 
generate at each iteration points in the graphs of all the monotone operators in such a way as to 
construct a half-space containing the Kuhn-Tucker set Z. The calculations of each of these points 
are resolvent computations involving a single monotone operator Ai or B^, which is what makes the 
methods splitting algorithms. The coordination step of the method is to project the current iterate 
onto the recently constructed half-space. The advantages of this approach are that it does not impose 
additional assumptions on the operators present in the formulation, it does not require knowledge of 
the norm of the linear operators (Tjfc)jg/or of combinations thereof, and it does not involve the 
inversion of linear operators. 

The methods of [1, 2] must evaluate all m + p resolvents of the operators (^i)ig/ and {Bk)k&K at 
every iteration, with only limited ability to pass information between these calculations. Essentially, 
the resolvents of all the operators [Ai)i^i must be evaluated independently, and then similarly for all 
the operators {Bk)k£K- In this setting, the only information flow within each iteration is from the 
(Aj)ig/ calculations to the {Bk)k&K calculations. This property results in an algorithm in which large 
blocks of calculations must be performed before any information is exchanged between subsystems. 
Although in principle conducive to parallel computing, this kind of structure can still lead to difficul¬ 
ties even in a parallel execution environment: it requires an essentially synchronous implementation, 
so if some small subset of the subsystems represented by the operators {Ai)i^i or {Bk)k&K are more 
computation-intensive than others, load balancing can become problematic: most processors may 
have to sit idle while the remaining few complete their tasks. This kind of structure is common to 
nearly all prior splitting schemes for more than two monotone operators, the only exception we are 
aware of being that of [20] for the special case 

find X eTi such that 0 E ^ BkX (1.7) 

keK 

of (1.1). In that case, information can flow in fairly arbitrary ways between the p resolvent calcula¬ 
tions comprising each iteration, as described by a set of algorithm parameters that is quadratic in p; 
the selection of these parameters is subject to a specific eigenvalue condition. However, the algorithm 
is still fundamentally synchronous, and it is has never been clear how to select its many parameters. 

This paper presents a different approach to constructing more flexible and potentially asyn¬ 
chronous decomposition methods for problems fitting the general structure represented by Prob¬ 
lem 1.1. The key idea is that our algorithm has the ability to process an essentially arbitrary subset 
of the operators between successive coordination/projection operations. The only restriction is one 
adapted from block-iterative methods for convex feasibility problems [6, 13, 23]: for some possibly 
large positive integer M, each operator must be processed at least once over every span of M consec¬ 
utive iterations. To our knowledge, this is the first application of this kind of versatile deterministic 
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control scheme to finding zeros of sums of operators. Such control schemes have been used in convex 
feasibility problems [6, 13]. This aspect of our algorithm gives it potential flexibility absent from 
other splitting schemes for monotone inclusions: first, it provides the ability to find an arbitrary bal¬ 
ance between computational effort expended on the subsystems and that expended on coordination. 
For example, if the subsystems are relatively time-consuming to process, one could perform as few as 
a single subsystem evaluation between successive projection steps, with the projections immediately 
spreading the information from each subsystem evaluation to each successive one. The second aspect 
of the flexibility of our approach involves the balance of computational effort between subsystems: in 
prior decomposition methods for monotone inclusions, every operator must be processed exactly the 
same number of times, but the class of algorithms proposed here is much more flexible. If, for exam¬ 
ple, some operators are less time-consuming to process than others, one has the option of processing 
them more frequently. Such features can be very useful in applications such as those described in [4] . 

Our analysis allows each activation of an operator to use information originating from an earlier 
iteration than the one in which its results are incorporated into the computation. This feature makes 
it possible to implement the algorithm asynchronously: the points in the graphs of the monotone 
operators incorporated into the projection step during a given iteration may be the results of resolvent 
computations initiated during earlier iterations. Our analysis shows that our method still converges 
so long as there is a fixed (but arbitrary) upper bound on the number of iterations between initiation 
and incorporation of a resolvent calculation. The potentially asynchronous nature of our method is a 
significant asset in the design of efficient parallel implementations. 

Prior work on projective splitting methods has used two different approaches to constructing affine 
half-spaces to separate the target set Z from the current iterate. The original approach in [19, 20] was 
developed for the inclusion problem (1.7). In this special case of (1.1), it was possible to efficiently 
confine the iterates to a specific subspace /C of the primal-dual space, which can be numerically 
advantageous. In the general setting of Problem 1.1, the analysis of [1, 2] used an alternative half¬ 
space construction in which the iterates are not confined to a subspace. A secondary contribution of 
this paper is to develop a unifying framework for constructing separators for Z in which both prior 
approaches appear as special cases. 

We present two classes of algorithms based on many of the same underlying building blocks and 
which may be viewed as asynchronous block-iterative extensions of the algorithms of [1, 2]. The first 
class uses a straightforward half-space projection at each iteration and allows for conventional overre¬ 
laxation of the projection steps by factors upper bounded by 2. This class exhibits weak convergence 
to an unspecified Kuhn-Tucker point. The second class is a variant that involves a more complicated 
projection operation and does not use overrelaxation, but induces strong convergence to the unique 
point in the Kuhn-Tucker set that best approximates a given reference point. Numerical experiments 
with these new algorithms are being conducted and we shall report on their results elsewhere. 

When applied in suitable product spaces, the block-coordinate methods of [16, 25] can be used 
to derive block-iterative splitting algorithms methods for a certain class of problems. However, unlike 
the methods we propose here, the resulting algorithms have been proved to converge only under 
random operator selection strategies, and they require either joint cocoercivity assumptions on the 
operators {Bk)k£K or the ability to block-decompose the projection onto the graph of certain linear 
operators. 

Notation. Our notation is standard and follows [8], which contains the necessary background on 
monotone operators and convex analysis. The scalar product of a Hilbert space is denoted by (• | •) 
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and the associated norm by || • ||. The projection operator onto a nonempty closed convex subset C of 
T-L is denoted by Pc- The symbols ^ and —> denote respectively weak and strong convergence, and 
Id denotes the identity operator. The Hilbert direct sum of two Hilbert spaces "H and Q is denoted by 
T-L®Q, and the power set of TL by 2^. Given A: V. ^ 2^, gra^ denotes the graph of A, A~^ denotes 
the inverse of A, and Ja = (Id +A)~^ denotes the resolvent of A. 


2 Analysis of a generic primal-dual composite inclusion problem 

2.1 Problem statement 

Our investigation will be simplified by the analysis of the following problem, which can be regarded 
as a reduction of Problem 1.1 to the case when m = K = 1, = 0, and ri = 0. 

Problem 2.1 Let P and Q be real Hilbert spaces. Let A: P ^ 2^ and S: ^ 2® be maximally 

monotone operators, and let L: ^ ^ ^ be a bounded linear operator. Consider the inclusion problem 

find X ^P such that 0 G Ax + L*BLx, (2.1) 

its dual problem 

find V* eg such that 0 € -LA-^{-L*v*) + S"V, (2.2) 

and the associated Kuhn-Tucker set 

Z = {{x,v*) eP®g \ -L*V* G Ax and Lx G P-^v*}. (2.3) 

The problem is to find a point in Z. The sets of solutions to (2.1) and (2.2) are denoted by ^ and 
respectively. 

Proposition 2.2 Consider the setting of Problem 2.1 and let Kbe a closed vector subspace ofPoQ such 
that Z c K. Then the following hold: 

(i) Z is a closed convex subset of x S'. 

(ii) 

(hi) For every a = (a, a*) G gmAand b = (6, b*) G graB, set s* b = {a* + L*b*,b — La), t* b = b^ 
%,b = (« I a*) + {b I b*), and 

= {x eK\ {x\ f* b) ^ %,b}- (2.4) 

Then the following hold: 

(a) Let a G gmA and b G grail. Then = K, s* b = ^ [{a,b*) e Z and ??a,b = O]. 

^ riaegravl (HbegraS “^ajb • 

(iv) Let (an,a*)neN be a sequence in gra^l, let (6„,6*)„gN be a sequence in grail, let x C P, and let 
V* G g. Suppose that a„ ^ x, 6* ^ v*, a* + L*6* —)• 0, and Lun — bn ^ 0. Then (x, v*) G Z. 
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Proof, (i): [12, Proposition 2.8(i)]. 

(ii) : [12, Proposition 2.8(iii)-(v)]; see also [24]. 

(iii) : For every a = (a, a*) G graA and b = (5,6*) G graB, set 


Ga,b = {x \{x\ S* b) ^ %,b}, 


(2.5) 


and observe that 


= {x&K\{x\ t* b) < ^a,b} 

= {x^K\{x \ PKsl^h) + I PK^^l,h) ^ ^a.b} 

= {x&K\{x\ S* b) ^ r/a,b} 

= K:nG3,b. 


( 2 . 6 ) 


(iii)(a): By [1, Proposition 2.2(i)], Ga,b = 'H®Q ^ a* b = 0 ^ («>('*) £ Z and r/a,b = 0. The 
claim therefore follows from (2.6). 

(iii)(b): By [1, Proposition 2.2(iii)] Z = flaGgrayi PlbegraB ^a,b • Hence, (2.6) yields Z = Kr^Z = 



(iv): [1, Proposition 2.4]. □ 

Remark 2.3 As will be seen in Remark 3.5, the subspace K in Proposition 2.2 adds flexibility to 
the implementation of our proposed algorithms when certain structures are present in the problem 
formulation. 

Proposition 2.4 Problem 1.1 is a special case of Problem 2.1. 

Proof. Let us set 


L: PL ^ Q: {xi)i^i i-)- ( LkiXi) 

A: PL ^ 2^ : {xi)i^i i—)• + AiXi) 

B: Q ^ 2^ : {yk)k&K 'Z.keKPkiVk ~ ^fc)- 


(2.7) 


Then 



( 2 . 8 ) 


With these settings, (2.1), (2.2), and (2.3) are respectively equivalent to (1.1), (1.2), and (1.3). □ 

2.2 A Fejer monotone algorithm 

We first recall some basic results concerning Fejer monotone sequences. 
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Proposition 2.5 [14] Let Kbe a real Hilbert space, let C be a nonempty closed convex subset of K, and 
let xq e K. Suppose that 

for n = 0 , 1, ... 

t* E /C and E M are such that C C Hn = {a; E /C | (a; | t*) ^ ??„} 

AnG]0,2[ 

®n+l — “1“ ^ni^PHn^n ®n)- 

Then the following hold: 

(i) (a;n)„gN is Fejer monotone with respect to C: (Vz E C)(yn E N) ||a;„+i — z|| ^ \\xn — z\\. 

(ii) ^^riGN An(2 ^n)\\PH ®n|| -|-00. 

(iii) Suppose that, for every x e K and every strictly increasing sequence (g'n)neN ht N, Xq^ ^ x ^ 
X e C. Then {xn)n& converges weakly to a point in C. 

Algorithm 2.6 Consider the setting of Problem 2.1 and let /C be a closed vector subspace of ^ ^ 

such that Z <zK. Let e E ]0,1[, let (xq, fo) G /C, and let (A„)„gN € [e, 2 — e]^. Iterate 

forn = 0 , 1 ,... 

(an,a*) E graA 
{K,hl) E gra.B 

(t* , tn) = PK{a*n + L*b*^, bn - LUn) 

'Cn = IldP + ll^np 

if r„ > 0 (2.10) 

9n = — maxjo, (x„ | 4) + {tn \ <) - (On I O - {bn \ 6n)| 

else 6n = 0 
a^n+l — 9ntn 

- '^n+1 ~ '^n~ ^ntri- 

Proposition 2.7 Consider the setting of Problem 2.1 and Algorithm 2.6, and suppose that ^ 0. Then 

the following hold: 

(i) {xn, 'yn)nGN is CL Sequence in K which is Fejer monotone with respect to Z. 

(ii) EnGN lkn+1 “ Xnf < +00 and Y^nm IK+1 - <IP < +00- 

(hi) Suppose that the sequences {an)neN> ibn)nm! {0‘n)neft, ctnd (5* )„gis} are bounded. Then 

lim {{Xn -an\a*n + L*Vn) + {Lxn -bn\b*n- <)) ^ 0. (2.11) 

(iv) Suppose that, for every (x, v*) E K and for every strictly increasing sequence {qn)nm in N, 

[ ^ X and v*^ ^ ^ (x,v*) E Z. (2.12) 

Then {xn)nm converges weakly to a point x G tP, (x*)„gN converges weakly to a point v* E 
and {x,v*) E Z. 
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Proof. Parts (i) and (ii) of Proposition 2.2 assert that Z is a nonempty, closed, and convex subset of 
K. Now set 


(VnEN) Xr, = (xn,v:), s*^ = islsn) = K + L*b*^,bn-Lan), = 

Vn = {an \a*^) + {bn \bl), and Hn = {x G K \ {x \ t*^) ^ r^n}- (2.13) 

Then it follows from (2.10) and Proposition 2.2(iii)(b) that (Vn E N) Z c Hn- Set (Vn E N) 
^n = ^f^Qnjy^n- Using [8, Example 28.16(iii)], we get 


(Vn E N) Pn^Xn = 

Hence, 


Xn + 


Xr. 


fin (®n I ^n) 4.* 


112 


if / 0 and {Xn I t;) > r?n 
otherwise. 


(2.14) 


(Vn E N) ^n — a^nW and Xn.\-1 — Xn T ^n{PHna^n (2.15) 

Therefore, we derive from Proposition 2.5 (ii) that 

^A2<+oo. (2.16) 

nCN 


(i) : This follows from (2.15) and Proposition 2.5(i). 

(ii) : We derive from (2.10) that 

(Vn E N) ||x„+i - + IK+i - = 9lrn = X^Al ^ 4Al (2.17) 

Hence, the claim follows from (2.16). 

(hi): Since HTtcII ^ !> (2.10) and (2.13) yield 

(VnEN) Tn = \\tlf 
< lls* iP 

= ||a; + L*6;|p + ||La„-6„|p 

^ 2(||a;f + ||L||2 ||5;||2 + ||Lf ||a„f + ||6„f). (2.18) 

Hence, (rn)neN is bounded. Therefore, since (2.16) implies that An 0 and since {xn)nm lies in K, 
we obtain 

(Vn E N) max{0, {{xn \ s*) + {sn | v*) - {an \ a^) - {bn | &n))} 

= max{0, {{xn I O - {an \ a*} - {bn | b*})} 

= max{0, ({xn I PK^n) - {an I <) - {bn I b^})} 

= max{0, ({xn I t*} + {tn I V*} - {an I a*} - {bn I b*})} 

— \fPnAn 

0. (2.19) 

Consequently, 

lim {fxn - On I a* + L* + (Lx„ - | 6* - v*f)) 

= lim [fXn I Sn) + (Sn | ^n) “ (On | a*f) - {bn \ ^n)) ^ 0. (2.20) 

(iv): This follows from (2.15) and Proposition 2.5(iii). □ 
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2.3 An Haugazeau-like algorithm 


Algorithm 2.6 produces sequences that converge weakly to some undetermined point in Z. We now 
describe an algorithm that provides strong convergence to the point in Z closest to some reference 
point (xq, fg) This approach relies on a geometric construction going back to [21] and was 

used in the context of Problem 1.1 in [2] . 

Let {x, y, z) G be an ordered triplet from a real Hilbert space K. We define 

H{x, ^) = € /C I {h — y \ X — y) ^ (2.21) 

and, if the set H{x,y) n H{y,z) is nonempty, we denote by Q{x,y,z) the projection of x onto it. 
The principle of the algorithm to project a point xq e K onto a nonempty closed convex set C C /C 
is to use at iteration n the current iterate Xn to construct an outer approximation to C of the form 
H{xo, Xn)r]H{xn, ®n+i/2); the Update is then computed as the projection of xo onto this intersection, 
i.e., Xn+i = (5(®0) ®n+i/ 2 )- As the following lemma from [21] shows, this last computation is 

straightforward; an alternative derivation may be found in [8, Corollary 28.21]. 

Lemma 2.8 ([21, Theoreme 3-1]) Let K be a real Hilbert space, let {x,y,z) G K^, and set R = 
H{x, y) n H{y, z). Further, set x = {x — y \ y — z), p = \\x — y\\^, u = \\y — z|p, and p = pu — x^- 
Then exactly one of the following holds: 


(i) p = 0 and x < 0, in which case R = 0. 

(ii) [p = 0 and x ^ 0] or p > 0, in which case Rf^0 and 


Q{x,y,z) 


z, if p = 0 and x ^ 0; 

^ x + {l + x/n){z - y), ifp > 0 and xn ^ p; 

.y + {n/p){x{x -y) + p{z - y)), ifp > 0 and xn < p. 


( 2 . 22 ) 


Proposition 2.9 ([2, Proposition 2.1]) Let Kbea real Hilbert space, let C bea nonempty closed convex 
subset of K, and let xq g K. Iterate 

for n = 0,1,... 

take Xn+i /2 G K. such that C C H{xn, ®n-i-i/ 2 ) (2.23) 

Xn+l — Q (®0) ®n-|-l/2) • 

Then the sequence {xn)nm tueZl defined and the following hold: 


(i) (Vn G N) \\Xn - ®o|| ^ \\Xn+l - Xoll ^ \\PcXo - ®o||- 

(ii) (Vn G N) C C H{xq, Xn) 0H{xn, a;n+i/ 2 ). 

(iii) EngN \\Xn+l -XnW^ < +00. 

(iv) EngN ll®n-rl/2 “ < +00. 

(v) Suppose that, for every x G K and every strictly increasing sequence ((?n)ngN tn N, ^ x ^ 
X G C. Then x^ —>• PcXo- 
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Algorithm 2.10 Consider the setting of Problem 2.1 and let /C be a closed vector subspace oi'H®Q 
such that Z c K. Let e G ]0,1[, let (xq, Xq) G K, and let (An)neN C [Ej 1]^- Iterate 


forn = 0,1,... 

(«n,an) e graA 
ibn,b*J G graB 

(t* , tn) = Pk{K + L*K^ bn - LUn) 

Tn = ll^nlP + PniP 
if r„ > 0 

0n = — maxjo, {Xn I tl) + {tn \ <) “ (On | <) “ {bn \ &n)| 
else On = 0 

®n+l/2 ~ ~ Ontn 

'^n+1/2 ~ '^n~ b^ntn 

_ (a;n+l,<+l) = Q{{xQ,vl),{Xn,V*n),{Xn+l/2,V*n+i/2))- 


(2.24) 


Remark 2.11 Using Lemma 2.8, the computation of the update {xn+i ><+i) in (2.24) can be explic¬ 
itly broken into the following steps: 


Xn = (XO - Xn I Xn - Xn+1/2) + {v^ “ < I < “ <+1/2) 

A^n = ||a:o - Xn\\‘^ + Ibo - 

= \\Xn - a:n+l/2f + ||< “ <+1/2^ 

Pn = Pnl'n “ Xn 

if Pn = 0 and Xn ^ 0 
Xn+l 3:n+l/2 
_ Xn+i = ^^n-rl/2 
if Pn P’ 0 and Xn^n ^ Pn 

Xn+l = Xo + (1 + Xn/T^n){Xn+l/2 “ Xn) 

_ <+i = + (1 + Xn/l^n)«+i/2 - V*n) 

if Pn ® 0 and Xn^n ® Pn 

^Tn+l — Xn “L {ynj Pn) (^Xn{xo Xn) ® M»i(®n+l/2 Xn)^ 
_ <+i =V*+ {iyn/Pn){Xn{Vo “ <) + Mn«+i/2 “ <))• 


(2.25) 


Proposition 2.12 Consider the setting of Problem 2.1 and Algorithm 2.10. Suppose that SA f <Z and 
set (xjV*) = Pz(xo,Vq). Then the following hold: 


(i) (2:n)neN cmd «)neN are bounded. 

® EngN W^n+l - XnP < +00 and J^neN IK+l “ < +00- 

fiii) EngN lkn+1/2 - Xnf < +00 Otld X^ngN ll<+l/2 “ <lP < +00- 

(iv) Suppose that the sequences (an)ngN, {bn)nm, (an)ri.gN. and (6n)ngN are bounded. Then 

lim ((Xn - On I a* + L*Vn) + (LXn -bn\bl- Vn)) ^ 0. (2.26) 

(v) Suppose that, for every (x, v*) G /C and every strictly increasing sequence {qn)n& in N, 

[xg„ ^ X and v*^ ^ x* ] ^ (x,v*) G Z. (2.27) 

Then (xn)ngN converges strongly toxCtS^ and (x*)ngN converges strongly to v* G 
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Proof. We first show that we recover the setting of Proposition 2.9 applied in /C to the set Z of (2.3), 
which is nonempty, closed, and convex by Proposition 2.2(i)-(ii). Set 


(Vn G N) Xn — [Xn,Vjf), ®n+l/2 — (®n+l/2)'^ri,+l/2)’ 

and = {an \ a^) + {bn \ h^). (2.28) 

If, for some n G N, we have Xn+ii 2 = Xn, then trivially Z c H{xn,Xn+i/ 2 ) = K.', otherwise, (2.24) 
imposes that {xn \ tf) > rfn and therefore that 

^ {Xn I ^n{{Xn \ ^n) 

{Xn I ^n) ^n'^n 

= {Xn - 9nt*n \ t^) 

= {Xn+1/2 I K), (2.29) 

from which we deduce using Proposition 2.2(iii) that 


Z C {a; G /C I I t*) ^ 

C I® G /C I (a; I tf) ^ {Xn+ll2 I ^n)} 

= I® G /C I (a; I Xn — XnJ^ijf) ^ (®n+l/2 I ~ ®n+l/2)} 

= H{xn, Xn+ 1 / 2 ) ■ (2.30) 


Altogether, (2.24) is an instance of (2.23) with C = Z, and we can apply Proposition 2.9. In particu¬ 
lar, Proposition 2.9(ii) asserts that {xn,Vn)nm is well defined. We can now establish the claims of the 
proposition as follows. 

(i) : This is a consequence of Proposition 2.9(i). 

(ii) : It follows from (2.28) and Proposition 2.9(iii) that^lncN lkrt+i-a:n|P + Z]nGN WK+i-KW^ = 

Xn\\ “t“00. 

(hi): In view of (2.28) and Proposition 2.9(iv), EneN lkn+1/2 - Xn\\‘^ + EneN ll'^n+1/2 

EnSN ll®n+l /2 “ < + 00 . 

(iv): Set (Vn G N) A„ = We derive from (2.24) and (iii) that 


nGN 


\2 

nGN ” nGN 


E 

nGN 


X 


n+1/2 


- Xr 


< + 00 . 


(2.31) 


The claim is then obtained by arguing as in the proof of Proposition 2.7(iii). 
(v): This follows directly from Proposition 2.9(v). □ 


Remark 2.13 Proposition 2.12 guarantees strong convergence to the projection of the initial point 
(xo,fo) onto the Kuhn-Tucker set under the same conditions that provide weak convergence to an 
unspecified Kuhn-Tucker point in Proposition 2.7. This phenomenon is akin to the weak-to-strong 
convergence principle investigated in a fixed-point setting in [7] . 
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3 Solving Problem 1.1 


3.1 Block iterations and asynchronicity 

In existing monotone operator splitting methods, each operator in the inclusion problem must be 
used at each iteration n in a resolvent calculation that must be based on information available at 
the current iteration. For instance, the methods of [1, 2] require points G gra^j and 

{bk,n, G graSfc for every i ^ I and every k ^ K, and these points must be computed using the 
current values of the primal variables {xi^n)i&i and of the dual variables {vl ^k&K- The earlier work 
in [19, 20] in the context of (1.7) is similar. The two main novelties we present in this paper are to 
depart from this approach by allowing asynchronous block iterations. Specifically, we allow: 

Block iterations: At iteration n, we require calculation of new points in the graphs of only some 
of the operators, say and {Bk)k^Kr,cK- The control sequences (/n)nGN and (iT„)neN dictate 

how frequently the various operators are used. 

Asynchronicity: A new point (ai,n,o*n) ^ graAj being incorporated into the calculations at 
iteration n may be based on data Xj ci(n) and c (n))^eir available at some possibly earlier iteration 
Ci{n) ^ n. Therefore, the calculation of {ai^n,a*n) cotild have been initiated at iteration Ci(n), with 
its results becoming available only at iteration n. Likewise, for every k G Kn, the computation of 
ibk,n, bl J G graBk can be initiated at some iteration 4(n) ^ n, based on {xi^dk(n))iei and 

To establish convergence, there needs to be some limits on the asynchronous asynchronicity lag of 
the algorithm and the spacing between successive calculations involving each operator, as described 
in the following assumption. 

Assumption 3.1 

(i) M is a strictly positive integer, {In)nm is cl sequence of nonempty subsets of I, and {Kn)neN is cl 
sequence of nonempty subsets of K such that 

( n+M—l n+M—1 \ 

IJ Ij = I and IJ ATj = a: . (3.1) 

j=n j=n J 

(ii) D is a positive integer and, for every i G I and every k G K, {ci{n))nm and {dk{n))nen are 
sequences in N such that 

(Vn G N) ^(Vi G /) n — D ^ Ci{n) ^ n and {\/k G K) n — D G, dk{n) ^ . (3.2) 

(hi) e G ]0,1[ and, for every i G I and every k G K, (7i,n)neN and {pk,n)nm are sequences in [e, 1/e]. 

At iteration n, our algorithms incorporates points in the graphs of the operators {Afjnzi^ and 
{Bk)keKn- Condition (3.1) ensures that over any span of M consecutive iterations, each operator 
is incorporated into the algorithm at least once. The standard case corresponds to using all the 
operators at each iteration, i.e. (Vn G N) In = I and Kn = K. Toward the other extreme, it is 
possible to use just one of the operators from and (Hfcjfcgic at iteration n. For example, such 
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a control regime could be achieved by setting M = max{m, p} and sweeping though the operators 
in a periodic manner. Condition (3.2) guarantees that the points in the graphs incorporated into 
the algorithm are based on information at most D iterations out of date. If the algorithm is being 
implemented synchronously, then one can simply set D = 0, in which case (Vn G N)(Vz G I)(\/k G K) 
Ci{n) = n and dk{n) = n. Finally, the positive scalars (7j,n)nGN and (/ifc,n)neN in (hi) are the proximal 
parameters used in the resolvent calculations. The assumption requires that they be bounded above 
and also away from 0. 

The following result is the key asymptotic principle on which our two main theorems will rest. 
The key idea of our algorithm is to simply recycle an old point in the graph of each operator for which 
new information is not available. 

Proposition 3.2 Consider the setting of Problem 1.1 and suppose that the following are satisfied: 


(a) For every i ^ I, {xi^n)nen is a bounded sequence in FLi and, for every k £ K, „)neN is a bounded 
sequence in Gk- 

(b) Assumption 3.1 is in force. 

(c) For every n G N, set 


for every i G In 

ii,n = J2keK ^kiK,Ci{n) 





for every i £ I \ In 
for every k £ Kn 

ik,n 

ibk,n, bk,n) — (fk + J r-k,d^(u)Bk {ik,n + Pk,dk{n)'^ k,dk(n) ~ (n) + Pk,dAn)^^^’'^ 

for every k £ K Kn 


bk,n)^ 

(3.3) 


and define 

(Vn G M) Un — j Ojn — ^ bn — {bk,n)kaK i and bn — ^k,n)kaK • (3.4) 


Then the following hold: 


(i) Define A and B as in (2.7). Then (Vn G N) (o„, a*) £ gra4. and (bn, b’f) £ graB. 

(ii) (cin)nsNj (o,n}'>T-&ti> and (bn)nGN tire bounded. 

(hi) Suppose that the following are satisfied: 

(d) (Vi G I) EnGN \\Xi,n+l “ < +00 and (Vk £ K) Yjn& \K,n+l - '^k,n\\^ < +00- 

(e) lim ( Yjiel {tti,n — ai,n \ n*,n + Y^k&K ^ki^k,v) + Y^k&K (Yliel ^kiXi,n — bk,n \ b% n~'^k,'n)) ^ 0. 
(0 {tln)nm is tL Strictly increasing sequence in N, for every i £ I, Xi £ TLi and ^ Xi, and, 

for every k£K,vl£Gk and vf 
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Then {{xi)i^i, {vDk^K) G Z. 


Proof. Define TL, Q, and L as in (2.7) and set 

(Vn G N) Xn = ixi^n)iei and u* = {vf^fk^K- (3.5) 

(i) : This follows from (3.3) and basic resolvent calculus rules [8, Propositions 23.15 and 23.16]. 

(ii) : Let i G I. We derive from hypothesis (a) and Assumption 3.1(iii) that the sequence (xj ci(n) ~ 

7i,ci(n) Efceir^L^fc,ci(n))nGN bounded. Since the operators (J 7 i,,.(„)AjneN are nonexpansive [8, 
Corollary 23.8], it follows from (3.3) that (aj^„)„gN is bounded, and hence that (o*,j)„gN is also 
bounded. Likewise, for every k e K, LkiXi^d^in) + is bounded and we 

deduce from (3.3) that (6fc,n)nGN and (6^,j)neN are bounded. In view of (3.4), this establishes the 
claim. 

(iii) : For every every i G / and every n G N, define ii (n) as the most recent iteration at which a 


new point in the graph of Ai was incorporated into the algorithm, that is, 

(Vz G /)(Vn G N) ii{n) = max{j G Sj | j ^ n}, where S* = G N | z G Ij}. (3.6) 

Note that (3.3) implies that 

(Vz G /)(Vn G N) {ai,n,alJ = (3.7) 

For every z G I, (3.1) yields sup„gj^(zz —lj(n)) ^ M and hence lim„^+oo ^i(n) = +(X). Next, we define 
(VzG/)(VnGN) £i{n) = Ci{ii{n)). (3.8) 

Thus, £i (n) is the iteration from which the computation of the most recent point in the graph of Ai 
was initiated. It follows from (3.2) that 

(Vz G /)(Vn G N) n — ii{n) = n — ii{n) + ii{n) — li{n) ^ M + D. (3.9) 


Hence, (Vz G I) lim„^+oo (n.) = +oo. Since maxjg/XljeN ll^*V+i “ < + 0 ° by (iii)(d), we 

deduce that 

j=ii{n) 

^{M + D) ^ \\xij+i-Xi^jf 

j=li(n) 

+ 00 

^{M + D) ^ \\xi,j+i - XijW^ 

j=ii{n) 

0. (3.10) 

Likewise, since (iii)(d) asserts that max^g;^ < +00, we have 

+00 

G I){'ik G K) + ^ (3.11) 

j=li(n) 
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Next, let us set 


(VzE/)(VnGN) < 




— \ 




'i,n {^i,£i{n) ^i,£i{n) 


keK 




+ E 


(n) 


fcG/t 


Then it follows from (3.7), (a), (ii), (3.10), and (3.11) that 


(Vi E /) Oi 


+ E ^*ki'^k,n 

keK 




In + E 


n) 


k£K 


— \ ^i.n ^i.n 


T.^*k^ivln-V*kMn)) 

k^K 


H“ ( ^2,n 


+ E ^*ki^kA(' 


keK 




n) 




E W^kiW sup (lixijil + llaijil) ) 

ieN ) 

+ ( SUp||o-j|| + V \\Lki\\sup\\vlj\\)\\Xi^n - Xi^i.(^n)\\ 

VieN jgN / 


0 . 


We also derive from (3.12), (3.7), and (3.3) that 


(Vi E /)(Vn E N) (/>,,„ = ( a:i, 4 (n) - at 




(n) 


fcGir 


_ -1 II _ ||2 




An+Y.^*k^vl,^{n) 

k€K 


which yields 


(Vi E I)(yTl E N) 0i,n ll^i,€i(n) ^ijnll' ^ 


Cli 


and 


(Vi E /)(Vn E N) (t)i^n = liMn) 


It follows from (3.15) that 


k&K 


+ E ^kiVk,li{n) ^ ^ + E ^*ki^kM 


(n) 


k&K 


(Vi E T)(yn E N) O-ijnll ^ + ||2^i,ti(n) ' 


< 


2(||Xj^,2 ®i/i(n)ll (</*»,n + £ 4^i,n) 


(3.12) 


(3.13) 


(3.14) 


(3.15) 


(3.16) 


(3.17) 
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and from (3.16) that 


(Vi E /)(Vn E N) 


+ ^ ^ki'^k 
k&K 

<n + E - E Ll^KAin) ” 




< 2 


^ 2 e 


fceir 


+ E 


keK 

2 


* 71 * 

ki^k,li{n) 


k&K 


+ Eiii 

keK 


11 2 11 * * 112 
fcdl ll^fc,£i(n) '^fc,nll 


i4’i,n + E! Il-^fc»ll W'^^kAin) '^k,n\\ V 

^ keK ^ 


(3.18) 


We now perform a similar analysis for the operators {Bk)k£K- Much as in (3.6), for every k £ K and 
every n E N, define dk{n) as the most recent iteration at which a new point in the graph of was 
incorporated into the algorithm, that is, 

{\/k E K){\/n E N) i9fc(n) = max{j E T*. | j ^ n}, where T*. = {j E N | A: E Kj], (3.19) 
and observe that 

(VA; E K)(yn E N) (6fc,n,6yn) = {hMn)^KMn))- 

Next, we define 

(VA; E iV)(Vn E N) ^k{n) = dk{^k{n)). (3.21) 

Then, we derive from (3.1) and (3.2) that 

(VA; E K)(yn E N) n — dkin) = n — ^k{n) + dk{n) — 'dk{n) ^ M + D (3.22) 

and therefore that (VA; E K) lim^^+oo ??fc(n) = +(X). Since maxjg/W 
(iii)(d), we then deduce that 


+ 00 


(VA: E iV) \\xi^n - ^ {M + D) E 11^2, j + 1 

3=^k{n) 


— 11 —^ 0* 


(3.23) 


Similarly since, max^g^ Y.jm ll^fcj+i '^k,j\\ < + 00 , we have 


+ 00 


(VA; E K) 

Now, let us set 


j=&k{n) 


A+l-vl^jf 0. 


(3.24) 


(VA; E iV)(VnE N) < 


'^k,n — ( ^ ^ I-'kiXi^n 


k,n 


iei 


bln - <n 


i^k,n — ^kiXiAkin) ^fc,i?fc(n) 


bkAkin-) 


(3.25) 


16 

















Then it follows from (3.20), (a), (ii), (3.23), and (3.24) that 

6 * * 

k,n 


(VA) G K) '4^k,n '^k,n — ^ ^ ^ 

'y ^ ^kiXi,'dk{n) 

''iei 

( y ^ Lki{Xi,n ~ Xi,'d).{n)) 


k,n 


' i&I 


T \ ^ ^ LkiXi,i)i.(n) ^k,n 


A* * 

^k,n 


5 * * 

k,n ^fc,i 


'^k,'dk{n) ^k,n 


< 


^||Lfci||sup(||6^j|| + ||r;^_J)") 
iGi ^ 

+ sup ( V IlLfcill + \\bk,j\\ ) 

ieN V / 


^i,n 




0 . 


' iG/ 


In addition, (3.20) and (3.3) yield 

(VA: E iT)(Vn E N) 

y ^ ^ki^i,'df^(n) ^k, 

iei 

~ f^k,'dk(n)\\^k,n ~ ''^k,'&i,{n)\\ ' 


A* * 

^k,n ^fe,i?j.(n) 


Consequently, 


(VAEiC)(VnEN) V’M = 


-A'fcia^i,i5t,(n) ^k,'dk(n) 
iel 

f^k,'&ii(n) 


> £ 


^ ^ ^kiXi^-/jl_(^n) ^k,'&i^(n) 

i&I 


and 


(VA: E iC)(Vn E N) V’;.,. = Mfc,^,(n)ll^fc,n - ^ 4K,n - 

It follows from (3.28) that 

_ 2 

(VA: E iC)(Vn E N) 


^kiXi^n bk ,ri 

^ ^ (xj^n ^ ^ ^fc,n 

iG/ iG/ 

^ ^ II-A/fctII ®i,'!?)i.(n) II “f ^ ^ ^ki^i,'&i.(n) ^fc,r; 

^ iG/ iG/ 

3 f ^ ^ ll-A'fci II ||2:i,n ^i,'&i;{n) II V £ (V’fc,?! '4^k,n) “f £ '4^k,n j j 

V r / 


(3.26) 

(3.27) 

(3.28) 

(3.29) 

(3.30) 
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and from (3.29) that 

(Vfe G i^)(Vn G N) \\bl^ - ^ 2{\\bl^ - + \\vl^ - 

^ 2(e {'lpk,n ~'>pk,n) ~\~ £ V’A:,n + H'^fc^n ~'^fc,i?fc(n) II )• (3.31) 
On the one hand, we derive from (3.5), (3.17), and (3.31) that 


^ ^ II Q-ijn II 


i&I 

keK 

2 ^ ^ ||2^i,n 

?i(n) P + 2 ^ \\Vk,n - '^k,iXk{n) 

i&I 

k&K 

+ 2e ^ 

4*i,n) T 2e ^ ^ {'4’k,n V^fc 


i&I 


k&K 


iG/ fcGit 

On the other hand, we derive from (3.5), (3.18), and (3.30) that 

(VnGN) ||o; + L*<f+ ||Lx„-5,,f 

2 

~ ®i,n + ^ki^k,n + ^kiXi^n ~ bk,n 

iel k&K k&K i&l 

^ 3e — 4‘i,n) +\\Lki\\ ll^fc/.(n) “'^fc,nll 

iG/ iG/ keK 

II ||2Ji,n ^i,i?j.(n)ll +26 ^ ^ {'lpk,n '^k,n) 

k£K 


(3.32) 


+ 2 ^ ^ ll-^fcj 
fcGtr i£l 


“f 26 f ^ ^ 'Pi,n “f ^ ^ '4^k,n 

^ iG/ feGit 


(3.33) 


We deduce from (3.12), (3.25), (3.4), (3.5), and (iii)(e) that 

lim ( ^ (t)i^n + ^ V'fc.n") = lim ((x„ - | o* + L*v*^) + (Lx^ - | 6* - u*)) ^ 0. (3.34) 


iG/ 


fcGit 


Altogether, taking the limit superior in (3.32) and (3.33), and using (3.10), (3.24), (3.13), (3.26), 
(3.11), (3.23), and (3.34), we obtain 


Xrfi — (Xfi —^ 0, (x^ “{“ L ^ 0, Lxji — b^ —y 0, and ^ 0. 

Now set X = {xi)i^i and v* = {v%)k£K- Then (iii)(f) and (3.35) yield 


^qn 


(3.35) 

(3.36) 


X, bl^^v\ a*„+L*5*^^0, and La,„-b,„^0. 

In turn, (i) and Proposition 2.2(iv) imply that (x, v*) G Z. □ 

Remark 3.3 In (3.3), the resolvents are assumed to be computed exactly to simplify the presenta¬ 
tion. However, it is possible to allow for relative errors in these computations in the spirit of [20, 
Algorithm 3]. More precisely, we can replace the calculation 




(3.37) 
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(3.38) 


by any choice of (ui^n, a*^) G Ti} such that 

iS^i^ni “t“ ^i^n) ^ and Cti^n “1“ ^i,Ci(n)^i^n ^i,Ci(n) Ti,Ci(n)^i,n ~f 

where the error ei^n satisfies 


II II ^ /3 

^ (ei,n I <n + lln) ^ f^7i.Ci(n) ||<n + 

. {^i,Ci{n) Oji,n \ ^ *^ll^i,Ci(n) ^i,n 


(3.39) 


for some constants /3 G ]0, +oo[ and a G ]0,1[ that are independent of i and n. It follows from [8, 
Proposition 23.21] that (3.38) can also be written as 


iS^i,niCLi^n) ~ (®*,Ci(n) “b 7i,Ci(n) (^i ^*,n) “b ; 7j^Ci(n) Q,i,n + Ci,n) h,n^ ■ (3.40) 

It may easily be seen that the calculations (3.37) satisfy (3.38) with Ci^n = 0, trivially fulfilling (3.39). 
In the setting of (3.39), (3.15) becomes 


(Vi G G N) (l)i^n — ®i,n|| + {^i,£i{n) 0,i,n \ 

^ ^(1 ^)\\^i,ii{n) Qi,n|| 


(3.41) 


and (3.16) becomes 

(Vi G /)(Vn G 1 


h,n — li,£i(n) 

^ e(l - a) 


In + E 


(n) 


k&K 


- e. 


<n+E^H<6( 

k&K 


<n+E^^*<7(n) 

keK 


(3.42) 


Likewise, we can replace the calculation 

{bk,n, bfc.n) = (j'k + Jk-k,dkin.)^k i}k,n + t^k,dk{n)'’^k,dk{n) ~ ’ ^k,dk{n) + l^k,dk{n)^^k,n “ fefc,n)^ (3.43) 

by any choice of {bk,n-, bl J G Gl such that 

{bk,n ~ ^ S^a and bk^n “L ^kk,dk{n)bk,n ^k,n + /^fc,d(i.(n)^A:,dfe(n) fk,ni (3.44) 

where the error fk,n satisfies 

’ ||/fc,n|| ^ b 

' {^k,n bk^n I fk,n} ^ CIKfc,n ^fc,n|| (3.45) 

Afk,n I - Vld.in)) ^ Cf^k,dUn)\\bl,n " ^M.(n)ll^ 

for some constants <5 G ]0,+cx3[ and ( G ]0,1[ that are independent of k and n. Altogether, the 
effect of such approximate resolvent evaluations is to replace e~^ by e“^(l — a)~^ or e“^(l — in 
(3.32)-(3.33), with the remainder of the proof of Proposition 3.2 remaining unchanged. 
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3.2 A weakly convergent algorithm for finding a Kuhn-Tucker point 


We propose a Fejer monotone primal-dual algorithm based on the results of Section 2.2 to find a point 
in the Kuhn-Tucker set (1.3). 

Algorithm 3.4 Consider the setting of Problem 1.1, let /C be a closed vector subspace of Hi 0 
®k&K such that Z c /C, and suppose that Assumption 3.1 is in force. Let (An)„gN G [e, 2 — let 

((xi,o)ie/, {vl Q)k&K) G /C, and iterate 


forn = 0,1,... 
for every i G 

K,n = ^ki^k,Ci(n) 

(Oi,n; O^i^n) ~ ('A'i,ci(n)0 {^i,Ci{n) + li,Ci{n){^i ~ h,n)) > Tj^ci(n) ~ 0,n) ~ 

for every i G / \ 

for every k G AT^ 

^k,n 'Yhial 

{bk,n, = [vk 0 (4,n + ^^k,dk{n)'^k4k{n) ~ ’’’k) ^'’^k,dk{n) f^k,dk{n)^^k,n “ 

: br every A: G A \ Kn 
|_ {bk,n,bl,^) = {bk,n-l,bl^^_i) 

; (bk,ri)k(zK^ 0 ^ kiPk,ri}^^^ ’’ (.bk,n ^ki^i,ri)kGK') 

'^n = Yli£l ll^i,nlP 0 YlkeK II^A:,n|P 


if Tn > 0 
On = 


maxi 0, ^ ^ {{xi^n I bi,n) {^i,n \ O^i^n)) 0 {{bk,n \ Xk^n) {bk,n \ bk^n)) 

I i&I k&K 


else 9n = 0 
for every i ^ I 
|_ Xi^n+l X-i ji ^nbi^n 
for every k & K 

L '^k,n+l ~ '^k,n ~ ^ntk,', 



(3.46) 


Remark 3.5 When Problem 2.1 has no special structure, on can take K = 0jg7 T~l-i 0 0fceir Gk iu 
Algorithm 3.4. In other instances, it may be advantageous computationally to use a suitable proper 
subspace K. For instance, if / = {1}, z\ = 0, and Ai: Tii ^ Hi is linear, then (1.3) reduces to 


Z = ^{xi,{v*k)k(^K) xieHi,Aixi + Y1 ^ki4 = 0, and 

k£K 

(\/k G K) Vk G Gk and LkiXi — Vk & (3.47) 


and we can use 


^ — i {xi, {vk)k£K) G Hi 0 Qk Aixi + L*kiVk — 0 

k&K keK 


(3.48) 
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In effect, this approach was adopted in [20] in the further special case in which Ai = 0 and (Vfc E K) 
Qk = 'Hi, Lki = Id, and = 0. 

Theorem 3.6 Consider the setting of Problem 1.1 and Algorithm 3.4, suppose that ^ 0, and let 

(Vn E N) Xn = {xi^nfei and v* = (vfJkeK- (3.49) 

Then {xn)nGN converges weakly to a point x £ {v^)nm converges weakly to a point v £ S', and 
{x,v*) £ Z. 


Proof. Define TL, Q, L, A, and B as in (2.7), and (an)neN, «)neN, (^n)nGN, and {bf)nen as in (3.4). 
Further, define (Vn E N) tn = {tk,n)kGK and t* = It follows from (3.46), (3.49), (2.8), and 

Proposition 3.2(i) that Algorithm 3.4 is a special case of Algorithm 2.6. Hence, upon invoking Propo¬ 
sition 2.4, we can apply the results of Proposition 2.7 in this setting. First, Proposition 2.7(i) implies 
that the boundedness assumption (a) in Proposition 3.2 is satisfied. Second, in view of (3.46), the 
sequence {an, a* )neN and {bn, (>n)neN are constructed according to assumption (c) in Proposition 3.2. 
We thus derive from Proposition 3.2(ii) that 

(®r),)nsN) {bn')n£N, and {bn)n£N ate bounded. (3.50) 

Furthermore, the summability assumption (iii)(d) in Proposition 3.2 is secured by Proposition 2.7(ii), 
while the limit superior assumption (iii)(e) in Proposition 3.2 holds by Proposition 2.7(iii). We there¬ 
fore use Proposition 3.2(iii) to conclude by applying Proposition 2.7(iv). To this end, take {x, v*) £ K 
and a strictly increasing sequence {qn)n€N in N such that Xq^ x and v*^ v*. Then Proposi¬ 

tion 3.2(iii) assert that {x,v*) £ Z. Thus, (2.12) is satisfied and the proof is complete. □ 


Remark 3.7 Theorem 3.6 subsumes [1, Theorem 4.3], which required the following additional as¬ 
sumptions; the implementation is synchronous, i.e., 

(Vn E N)(Vi E /)(VA: E a:) Ci{n) = dk{n) = n, (3.51) 

no proper subspace is used, i.e., 

K: = 07f,©0gfc, (3.52) 

iei keK 

the control is fully parallel, i.e., 

(Vn E N) In = I and Kn = K, (3.53) 

and common proximal parameters are used in the sense that 

(Vn E N)(Vi E /)(VA: E AT) yi,n = In and Pk,n = hn- (3.54) 

Therefore, the proposed method also subsumes [18] and [19, Proposition 3] (see also [5]), which are 
special cases of [1, Theorem 4.3]; see [1, Examples 3.7 and 3.8] for details. 

Remark 3.8 Theorem 3.6 is closely related to [20, Proposition 4.2] (see also [5]), which considers 
the special case of Problem 1.1 in which I = {!}, z\ = 0, Ai = 0, and (VA: £ K) = TLi and 
Afci = Id. If in this case one sets 


K 


(xi, {vf)k£K) £ 




k&K 


(3.55) 
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in our algorithm, we recover the special case of the method of [20, Section 4] in which the parameter 
a^j of [20, Proposition 4.2] is 1 if i = j, and 0 otherwise. Other settings of in [20] produce 
algorithms that are not special cases of our scheme, but must process the resolvent of every operator 
at every iteration and remain fully synchronous as in (3.51) and (3.53). 

Remark 3.9 Recall that the resolvent of the subdifferential of a proper lower semicontinuous con¬ 
vex function f: V. ]—oo,-|-oo] is Moreau’s proximity operator {Id+df)~^ = prox^: x i-)- 
argminj^g.^(/(y) + ||x — y|P/2) [8, 22]. Now consider the setting of Problem 1.2 and execute Al¬ 
gorithm 3.4 with (Vi G I) Ai = dfi and (VA: G K) = dg^- Then, using the same arguments as 
in [15, Proposition 5.4], it follows from Theorem 3.6 that {xn)nm converges weakly to a solution to 
(1.5) and that (f^)neN converges weakly to a solution to (1.6). 

Remark 3.10 The framework of [20, Algorithm 3] for solving (1.7) allows for relative errors in the 
computation of the resolvents. Similar errors may be incorporated in Algorithm 3.4 by adopting the 
approximate evaluation scheme of Remark 3.3 to select points in the graphs of the monotone opera¬ 
tors in (3.46). Since Proposition 3.2 remains valid with such approximate resolvent computations, so 
does Theorem 3.6. 


3.3 A best approximation result 

In this section we use the abstract Haugazeau-like algorithm of Section 2.3 to devise a strongly con¬ 
vergent asynchronous block-iterative method to construct the best approximation to a reference point 
from the Kuhn-Tucker set (1.3). 

Algorithm 3.11 Consider the setting of Problem 1.1, let /C be a closed vector subspace of 0^^/ Tii © 
such that Z c K, and suppose that Assumption 3.1 is in force. Let (A„),igN C let 

{{xifi)i^i, {vl Q)keK) G K., and iterate 
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for n = 0,1,... 
for every i £ In 

^i,n = ZlfcGir ^ki^k,Ci{n) 

0-i^n) ~ (^'^'yi,ci(n)Ai{^i,Ciin) +7i,Ci(n)(^i 

for every i G / \ 




for every A: G Kn 

lk,n ^kiXi^df^^n) 

ibk,n,bl,n) — {j'k + '^lik,df,{n)Bk{^k,n + lJ'k4k{n)'^k,dk{n) “("-) f^fc,dfc(n) * 

:br every k € K \ Kn 

|_ {bk,n,bl ,^) = (6fc,n-l) 

((^i,n)*G/) {tk,n)k£K) = ^Kiifii^n “f YlkeK ^kibk,n)i&I^ {bk,n ~ Y2iel ^kiO‘i,n)kGK) 

"Ti = XliG/ ll^i.nP + YlkGK 

if Tn > 0 


e'n = — max 


else 6n = 0 


0) ^ ^ (y{Xi,n I ti^n) {(ki,n \ Q'i,n)) “f ^ ^ (y{tk,n \ '^k,n) {bk,n \ b 
i&I k£K 


for every A G / 

|_ ‘^i,n+l/2 Xi,n ~ ^riti^n 

for every k & K 


[ ^fc,n+l/2 ~ ^k,n bntk,n 

Xn = Eiel {Xifl - Xi,n I Xi,n - Xi^n+l/2) + Ek&K (^fc,0 “ ^k,n I ^fc,n “ ^fc,n+l/2) 
k'n = ZliG/ lk*,0 — iCi,n|P + J2k£K H^fc.O “ '^fc,nll^ 

= 'Zidl \\Xi,n - Xi^n+l/2f + EkeK \K,n “ ^fc,n+l/2ll^ 

Pn — Pnk'n ~ Xn 
if Pn = 0 and Xn^O 
for every i G / 

|_ Xi^n+l Xi^n+1/2 

for every k £ K 

* _ ^ 

Xk^n+l ^fc,n+l/2 

if Pn > 0 and Xn^n ^ Pn 
for every A G / 

|_ Xi^n+l — Xi^Q + (1 + Xn/k'n^{Xi^n+l/2 Xi^n) 

for every k € K 

[ ^In+l = + (1 + Xnlj^n){vln+l/2 “ 4,n) 

if Pn > 0 and Xnt'n < Pn 
for every A G / 

|_ Xi^n+l — Xi^n “f {yn/Pn)iyXn(Xi,0 Xi^n) “f pn{Xi n+l/2 ^i,n)) 

for every k £ K 

^k,n+l = Vk,n + i’^n/pn){Xn{vlo - + Pnivl n+1/2 “ ^k,n)) ' 



Theorem 3.12 Consider the setting of Problem 1.1 and Algorithm 3.11, and suppose that SA tZ. 
Define 

(Vn E N) Xn = ixi,n)iei and v* = (vfJkeK (3.57) 

and set {x,v*) = Pzixo,VQ). Then {xn)nm converges strongly toxC^ and {v^)neN converges strongly 
to V* E &. 

Proof Define TL, G, L, A, and B as in (2.7), (a„)neN, (OneN, (ftn)neN, and (6* )„eN as in (3.4), and 
set (Vn £N)tn = {tk,n)k£K and In view of (3.56), (3.57), (2.8), and Proposition 3.2(i), 

Algorithm 3.11 is an instance of Algorithm 2.10. Hence, upon invoking Proposition 2.4, we can ap¬ 
ply the results of Proposition 2.12 in this setting. First, Proposition 2.12(i) implies that assumption 
(a) in Proposition 3.2 is satisfied. Second, in view of (3.56), assumption (c) in Proposition 3.2 is 
satisfied as well. Thus, Proposition 3.2(ii) asserts that the sequences (an)nGNj (an)nGNj {bn)nen, and 
(&n)neN are bounded. Third, assumption (iii)(d) in Proposition 3.2 is secured by Proposition 2.12(ii). 
Finally, assumption (iii)(e) in Proposition 3.2 holds by Proposition 2.12(iv). We therefore use Propo¬ 
sition 3.2(iii) to conclude by invoking Proposition 2.12(v). Take {x,v*) E K and a strictly increasing 
sequence {qn)nm in N such that Xq^ x and v*^ v*. Then it follows from Proposition 3.2(iii) 

that {x,v*) E Z, which completes the proof. □ 

Remark 3.13 As in Remark 3.9, consider the setting of Problem 1.2 and execute Algorithm 3.11 with 
(Vi E /) Aj = dfi and (V/c C K) = dg^. Then Theorem 3.12 asserts that (x„)„gN converges 
strongly to a solution x to (1.5) and that {v^)n£N converges strongly to a solution v* to (1.6) such 
that {x,v*) is the projection of (xq, Uq) onto the corresponding Kuhn-Tucker set (1.3). 

Remark 3.14 Theorem 3.12 improves upon [2, Proposition 4.2], which addresses the special case in 
which the algorithm is synchronous and the restrictions (3.51)-(3.54) are imposed. The latter was ap¬ 
plied in the context of Remark 3.13 to domain decomposition methods in [4]; Theorem 3.12 provides 
a new range of ways to revisit such applications using asynchronous block-iterative calculations. 

Remark 3.15 By an argument similar to that of Remark 3.10, Theorem 3.12 remains valid if the 
resolvent computations in (3.56) are replaced by approximate evaluations meeting the conditions in 
Remark 3.3. 
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